[PB] Traverse2.0

支持版本：3.50.0

简介

遍历全表数据。支持Limit参数控制每次遍历返回的记录数，同时返回响应包中包含TraverseSession字段和Complete字段，遍历条件：

Limit: 控制每次遍历返回的记录条数
TraverseSession: 字符串，初始值: 0.0, 用于每次遍历判断遍历位置，类似于offset作用，本次返回的Session值，传入下次遍历Data参数中
Complete: bool, 取值范围：False 或 True, 用于判断记录是否遍历完成
Select: 指定每次遍历返回的字段



POST http://{Tcaplus_REST_URL}

请求语法

Http请求


#国内idc&dev&其他自建环境：联系dba分配restproxy进程后由dba提供相应Tcaplus_REST_URL
#腾讯云控制台　RESTful Endpoint, ip:80, 端口默认80
http://172.17.0.22
#TcaplusDB本地docker版 RESTful Endpoint, ip:31001, 端口默认31001
http://9.135.8.93:31001

Http头

名称	是否必填	限制条件	说明
x-tcaplus-target	是	无	Tcaplus.Traverse
x-tcaplus-version	是	无	Tcaplus3.50.0
x-tcaplus-app-id	是	无	对应业务id编号(aka,集群接入id)
x-tcaplus-zone-id	是	无	对应zone编号(aka,表格组id)
x-tcaplus-protocol-version	是	无	对应protocol版本号,默认2.0
x-tcaplus-table-name	是	无	对应表名
x-tcaplus-pwd-md5	是	无	业务密码(aka,集群访问密码)，传入计算后的md5值
x-tcaplus-idl-type	是	无	protobuf
x-tcaplus-result-flag	否	无	0：操作成功后不返回数据1：操作成功后返回和请求一致的数据2：操作成功后返回本次update操作后的数据3：操作成功后返回tcapsvr端操作前的数据
x-tcaplus-data-version-check	否		1：检测记录版本号,只有当该版本号与服务器端的版本号相同时，该版本号才会自增2：不检测记录版本号，强制把客户端的记录版本号写入到服务器中3：不检测记录版本号，将服务器端的版本号自增
x-tcaplus-data-version	否		具体的version值

示例：


x-tcaplus-target:Tcaplus.Traverse
x-tcaplus-app-id:3
x-tcaplus-zone-id:1
x-tcaplus-protocol-version:2.0
x-tcaplus-table-name:game_players
x-tcaplus-pwd-md5:4e81984efccfb4982333aeb1ff7968d5
x-tcaplus-result-flag:2
x-tcaplus-version:Tcaplus3.50.0
x-tcaplus-data-version-check: 3
x-tcaplus-idl-type:protobuf

Data

使用json格式表示记录相关信息。参数说明：

Limit: 必选，限制每次遍历返回的数据条数
ReturnValues: 可选，用户缓存，用于一些全局对象的传递，避免内存占用管理
TraverseSession: 必选，用于判断遍历循环Session的状态，类似于Offset概念，从响应包的TraverseSession中
Condition：可选，指定条件遍历，如”Condition”: “x > 1”。
Select: 可选，指定返回的记录字段

说明：早期版本的 restproxy 中该字段拼写为 TraveseSession，现已修正为 TraverseSession。当前版本请求中两种写法都支持，响应中会同时返回 TraverseSession 和 TraveseSession（取值相同），建议统一使用 TraverseSession。

示例：


{
"Limit": 1000,
"TraverseSession": "0.0",
"ReturnValues": "user buffers, to build a bridge between request and response, use case: global object that do not want to save in memory"
}

Http响应包大小限制： restproxy服务为每个链接配置了10MB的发送缓冲区，如果单个Http响应接近10MB，末尾的记录只会返回Key；用户需要通过Limit限制响应的记录数，从而规避这个限制
比如：


{
"ErrorCode": 0,
"ErrorMsg": "Succeed",
"ReturnValues": "user buffers, to build a bridge between request and response, use case: global object that do not want to save in memory",
"MultiRecords": [{
"RecordVersion": 1,
"Record": {
"player_id": 3,
"player_name": "1",
"player_email": "15",
"game_server_id": 1,
"login_timestamp": [],
"logout_timestamp": [],
"is_online": false,
"pay": {
"pay_id": 0,
"amount": 0,
"method": 0
}
}
}, {
"RecordVersion": 1,
"Record": {
"player_id": 4,
"player_name": "44",
"player_email": "44",
"game_server_id": 44,
"login_timestamp": ["123456", "234"],
"logout_timestamp": ["123456", "234"],
"is_online": true,
"pay": {
"pay_id": 44,
"amount": 44,
"method": 44
}
}
},

// 如果上面两条记录总大小接近10MB，则后面的记录只返回key

{
"RecordVersion": 1,
"MoreKeys": {
"player_id": 54,
"player_name": "54",
"player_email": "54"
}


}, {
"RecordVersion": 3,
"MoreKeys": {
"player_id": 4,
"player_name": "4",
"player_email": "4"
}
}, {
"RecordVersion": 7,
"MoreKeys": {
"player_id": 5,
"player_name": "5",
"player_email": "5"
}
}],
"TotalNum": 5,
"Complete": true,
"TraverseSession": "80614.48569856"
}

完整请求示例

一次性遍历示例

Limit设置比较大，一次就返回所有记录。适合表记录数小的场景


curl -i -XPOST -H 'x-tcaplus-target: Tcaplus.Traverse' -H 'x-tcaplus-app-id: 70' -H 'x-tcaplus-zone-id: 1' -H 'x-tcaplus-protocol-version: 2.0' -H 'x-tcaplus-table-name: game_players' -H 'x-tcaplus-pwd-md5: 0972ad76decf4d11a69e2e0d9af335da' -H 'x-tcaplus-result-flag: 2' -H 'x-tcaplus-version: Tcaplus3.50.0' -H 'x-tcaplus-data-version-check: 1' -H 'x-tcaplus-idl-type: protobuf' http://172.17.32.17 -d '{
"Limit": 1000,
"TraverseSession":"0.0",
"Select": ["gamesvrid"],
"ReturnValues": "user buffers, to build a bridge between request and response, use case: global object that do not want to save in memory"
}'

循环遍历示例

Limit设置适中，同时结合TraverseSession和Complete来进行循环遍历判断。以Bash脚本举例：


#!/bin/bash
PROTO_VERSION="2.0"
TCAPLUS_VERSION="Tcaplus3.50.0"
IDL_TYPE="protobuf"
CHECK_POLICY=3
result_flag=2
rest_url="http://172.17.32.17"
app_id=70
zone_id=1
table_name="game_players"
passwd=""

#caculate the md5 value of password
md5_passwd=$(echo -n $passwd | md5sum | cut -d ' ' -f1)

function build_headers(){
headers=(
-H "x-tcaplus-target: ${1}"
-H "x-tcaplus-app-id: ${app_id}"
-H "x-tcaplus-zone-id: ${zone_id}"
-H "x-tcaplus-protocol-version: ${PROTO_VERSION}"
-H "x-tcaplus-table-name: ${table_name}"
-H "x-tcaplus-pwd-md5: ${md5_passwd}"
-H "x-tcaplus-result-flag: ${result_flag}"
-H "x-tcaplus-version: ${TCAPLUS_VERSION}"
-H "x-tcaplus-data-version-check: ${CHECK_POLICY}"
-H "x-tcaplus-idl-type: ${IDL_TYPE}"
)
}
function build_traverse_loop_data(){
cat <<EOF
{
"Limit": 1,
"TraverseSession": "$1",
"ReturnValues": "user buffers"
}
EOF
}
function traverse_loop() {
target="Tcaplus.Traverse"
build_headers "$target"
complete="False"
limit=1
traverse_session="0.0"
total_num=0
while [ "$complete" == "False" ];do

ret=$(curl -i -XPOST "${headers[@]}" "$rest_url" -d "$(build_traverse_loop_data $traverse_session)" | grep -v HTTP | grep -v "json-1.0" | grep -v "content-length" | grep -v "^$" )

echo "$ret"

#获取complete值，False 或true, False: 表示数据尚未结束，true:表示遍历结束．借助Python３的Json
complete=$( echo "$ret" | python3 -c "import sys, json; print(json.load(sys.stdin)['Complete'])" )
　　
　　　#获取TraverseSession值，作为下一次遍历定位使用，类似于Offset作用.
traverse_session=$( echo "$ret" | python3 -c "import sys, json; print(json.load(sys.stdin)['TraverseSession'])" )

if [ "$complete" == "False" ];then

total_num=$(($total_num+1))


fi

done
echo "TotalRecords:$total_num"
}

declare -a headers

#traverse
traverse_loop

返回语法

返回参数说明

参数名	说明
ErrorCode	返回码
ErrorMsg	返回信息
ReturnValues	用户设置的保留数据，随请求到达 tcaplus 并由应答原样带回
MultiRecords	json格式，本次遍历返回的记录集合
TotalNum	本次遍历返回的记录条数
Complete	bool类型，False表示数据尚未结束，True表示遍历结束
TraverseSession	本次遍历返回的Session游标值，传入下次遍历Data参数中
TraveseSession	旧版本字段名，与TraverseSession取值相同，仅为兼容保留

返回示例

判断Complete参数是否为true, 如果为false, 需要通过循环遍历方式继续请求。


{
"ErrorCode": 0,
"ErrorMsg": "Succeed",
"ReturnValues": "user buffers, to build a bridge between request and response, use case: global object that do not want to save in memory",
"MultiRecords": [{
"RecordVersion": 1,
"Record": {
"player_id": 3,
"player_name": "1",
"player_email": "15",
"game_server_id": 1,
"login_timestamp": [],
"logout_timestamp": [],
"is_online": false,
"pay": {
"pay_id": 0,
"amount": 0,
"method": 0
}
}
}, {
"RecordVersion": 1,
"Record": {
"player_id": 4,
"player_name": "44",
"player_email": "44",
"game_server_id": 44,
"login_timestamp": ["123456", "234"],
"logout_timestamp": ["123456", "234"],
"is_online": true,
"pay": {
"pay_id": 44,
"amount": 44,
"method": 44
}
}
}, {
"RecordVersion": 1,
"Record": {
"player_id": 54,
"player_name": "54",
"player_email": "54",
"game_server_id": 54,
"login_timestamp": [],
"logout_timestamp": [],
"is_online": false,
"pay": {
"pay_id": 54,
"amount": 54,
"method": 54
}
}
}, {
"RecordVersion": 3,
"Record": {
"player_id": 4,
"player_name": "4",
"player_email": "4",
"game_server_id": 4,
"login_timestamp": ["123456", "234"],
"logout_timestamp": ["123456", "234"],
"is_online": true,
"pay": {
"pay_id": 4,
"amount": 4,
"method": 4
}
}
}, {
"RecordVersion": 7,
"Record": {
"player_id": 5,
"player_name": "5",
"player_email": "5",
"game_server_id": 6,
"login_timestamp": ["123456", "234"],
"logout_timestamp": ["123456", "234"],
"is_online": false,
"pay": {
"pay_id": 5,
"amount": 5,
"method": 5
}
}
}, {
"RecordVersion": 1,
"Record": {
"player_id": 5,
"player_name": "5",
"player_email": "55",
"game_server_id": 55,
"login_timestamp": [],
"logout_timestamp": [],
"is_online": false,
"pay": {
"pay_id": 55,
"amount": 55,
"method": 55
}
}
}, {
"RecordVersion": 5,
"Record": {
"player_id": 6,
"player_name": "6",
"player_email": "6",
"game_server_id": 6,
"login_timestamp": ["123456", "234"],
"logout_timestamp": ["123456", "234"],
"is_online": true,
"pay": {
"pay_id": 6,
"amount": 6,
"method": 6
}
}
}],
"TotalNum": 7,
"Complete": true,
"TraverseSession": "80614.48569856"
}

错误码

参考常见错误码

其它参考文档

[PB Generic表][C++ SDK]遍历表数据接口说明

[PB Generic表][Go SDK]遍历表数据接口说明

TcaplusDB用户手册

[Generic表]遍历表中的数据